Validation of the Barthel Index as a Predictor of In-Hospital Mortality among COVID-19 Patients

In order to predict the high mortality due to COVID-19, simple, useful and remote instruments are required. To assess the validity of the baseline Barthel Index score as a predictor of in-hospital mortality among COVID-19 patients, a validation study of a clinical prediction tool in a cohort of patients with COVID-19 was conducted. The primary variable was mortality and the Barthel Index was the main explanatory variable. Demographic, clinical and laboratory variables were collected. Other mortality predictor scores were also assessed: Pneumonia Severity Index, CURB-65 and A-DROP. The Receiver Operating Characteristic Area under the Curve (ROC AUC), sensitivity and specificity were calculated for both the Barthel Index and the other predictor scores. An analysis of the association between the main variables was conducted, adjusting by means of three multivariate models. Three hundred and twelve patients were studied. Mortality was 16.4%. A mortality Odds Ratio (OR) of 5.95 was associated with patients with a Barthel Index ≤ 90. The model number 3 was developed to predict in-hospital mortality before COVID-19 infection occurs. It exhibits an OR of 3.44, a ROC AUC of 0.792, a sensitivity of 74.5% and a specificity of 73.9%. The Baseline Barthel Index proved useful in our population as a predictor of in-hospital mortality due to COVID-19.


Introduction
Between January and December 2019, the World Health Organisation (WHO) confirmed 41 cases of an infection by a new coronavirus, called SARS-CoV-2, in the city of Wuhan, China [1,2]. On 11 March 2020, the WHO reported 118,000 cases (4291 deaths) in 114 countries and characterised the situation as a pandemic [3]. The spread of the virus continued and one year later, in March 2021, the number of cases had risen to 116,736,437 and deaths to 2,593,285 worldwide [4].
The disease, referred to as COVID-19, presents with a fever, sometimes with dyspnoea and bilateral pulmonary infiltrates [5]. Since the beginning of the pandemic, several studies sought to establish the typical profile of patients affected by COVID-19 [6], as well as clinical and analytical indicators that could be predictors of mortality due to this disease [7]. One of the first findings was the association between advanced age and mortality due to COVID-19. The meta-analysis by Zheng et al. [7] found a six-fold increased risk in patients over 65 years of age.
At present, there are validated tools for estimating mortality in different types of pneumonia, including SARS-CoV-2-related pneumonia [8], such as the Pneumonia Severity Index (PSI) [9], the CURB-65 [10] score and the A-DROP score [11]. The disadvantage of these tools is that one must wait for the individual to be admitted to hospital for different

Study Design
A prospective observational validation study in a cohort of adult patients admitted with COVID-19 to a public third-level hospital.

Sample/Participants
Patients ≥ 18 years old diagnosed with COVID-19 pneumonia who were admitted to hospital wards were included. Patients were excluded if they had a positive PCR (Polymerase Chain Reaction) test or were positive for antigens, but did not have a radiological diagnosis of pneumonia.
Based on the incidence of in-hospital mortality in the SEMI-COVID-19 registry (17.5% in a sample of 15,111 patients) [20], a confidence level of 95%, an accuracy of 4.50 units and an expected loss of 0.5%, the minimum sample size was estimated to be 276 patients. The replacement rate proved to be higher in the end, so the recruitment time was prolonged so that the statistical power would not be decreased.

Data Collection
The data were collected by nurses from hospital wards service. To avoid systematic errors in the data collection and the application of the scores, the healthcare professionals who collaborated on the data collection were previously instructed and trained. Recruitment was conducted between November 2020 and February 2021.

Primary Outcome
The primary outcome was to identify the incidence of the in-hospital mortality among adult COVID-19 patients and assess their prognosis on the basis of the basal BI.
The variable in-hospital mortality due to COVID-19 was obtained by following up each patient during their entire hospital stay; its outcome was dichotomised (yes/no).
The BI is obtained by asking 10 questions, the answers to which lead to a score between 0 and 100 points. The higher the score, the greater the independence in the performance of BADL [15]. In addition, different degrees of dependency can be established: total dependence (0-20 points), severe dependence (21-60 points), moderate dependence (61-90 points), slight dependence (91-99 points), and independent (100 points) [21].
The BI was applied as soon as possible after the COVID-19 diagnostic with the patients being oriented to answer based on their experiences before the diagnostic. If the patient had cognitive impairment or problems with effective communication, the BI was assessed through an interview of a close relative or primary caregiver [22].

Secondary Outcome
The secondary outcome was to compare the prediction of COVID-19 mortality by the pre-infection BI to other validated tests that require invasive testing (PSI, CRUB-65 and A-DROP).
The PSI is composed of 19 items of a different nature: sociodemographic parameters, comorbidities and analytical parameters [9]. Higher scores indicate a higher probability of death.
CURB-65 is an acronym for the parameters that comprise the instrument: Confusion, Urea, Respiratory rate, Blood pressure and age > 65. Each of these parameters is assessed dichotomously. Each question scores 1 point, so the result ranges from 0 to 5 points. Increasing scores indicate an increased risk of mortality [10].
The A-DROP tool is a modification of CURB-65 and is very similar to its parent scale. It is also evaluated based on a score ranging from 0 to 5 points [11].
The Charlson comorbidity index is a marker for chronic disease burden that helps predict the ten-year mortality for a patient who may have a range of comorbid conditions. The scores range from 0 to 33, with higher scores indicating a greater burden of chronic illness.
The confusional state was estimated by qualitative perception by healthcare workers. Three questions were asked: Do you know what day it is today? Do you know where you are? Do you know what happened to you? If the patient fails in all questions, he/she is considered confused.
All these variables were collected at the time of patient admission except for this last one.
(b) Index and scores: during the hospital stay, the following scores for estimating COVID-19 mortality were collected: PSI, CURB-65 and A-DROP. In addition, the baseline BI was obtained prior to hospital admission.

Ethical Considerations
Approval from the ethical and clinical research committee of the centre was obtained (Code 30/2020).

Data Analysis
Categorical variables were expressed as a frequency and percentage. The chi-square (χ 2 ) or Fisher's test was used for between-group comparisons. Normality was tested using the Kolmogorov-Smirnov test. Quantitative variables were expressed as the mean and standard deviation for normal distributions and median and interquartile range for non-normal distributions, and groups were compared using the Student's t-test or Mann-Whitney test, as appropriate. Odds Ratios (ORs) with their respective confidence intervals were also obtained.
The sensitivity, specificity, positive and negative likelihood ratios (LR+ and LR−) and accuracy were calculated for the best cut-off points of the BI, PSI, CURB-65 and A-DROP. Their respective Receiver Operating Characteristic (ROC) curves were plotted and the different Areas Under the Curve (AUCs) with their confidence intervals were compared.
For the multivariate logistic regression analysis, all patient-related variables described above were used except for days of hospital stay. The variables were entered separately together with the BI and the variation of the model was observed. For the design of the possible models, the following criteria were followed: accuracy (OR for the BI with the narrowest 95% confidence intervals); no requirement for invasive techniques to obtain the result of the variable; and the consideration of socio-demographic factors, including at least age and gender. The final validation of the models was carried out using the Hosmer-Lemeshow goodness-of-fit test, and the predicted probabilities were represented by the ROC curves and their AUCs.
Some of the continuous variables were dichotomized. Their cut-off point was set using the Youden index [23] for the maximization of sensitivity and specificity in the classification of the primary outcome.
Individuals with missing data in their variables that prevented the calculation of all predictor scores (BI, PSI, CURB-65 or A-DROP) were excluded from the analysis.
For all comparisons, a statistical significance level of p < 0.050 was established. The data were analysed using the IBM SPSS Statistics software Version 24.0.

Validity and Reliability/Rigour
All values and behaviours were measured by scoring items from the International Council of Nurses Code of Ethics for Nurses, 2021 revised English version.

Results
The data of 312 patients were analysed. A flow diagram with the patients' progression through the study is provided in the Figure 1.

Patient Characteristics
The mean age of the patients was <65 years, the majority of them (54.2%) were men. Thirty-six patients (11.5%) were institutionalised prior to hospital admission and the median Charlson comorbidity index score was 1 [0-2]. The most prevalent chronic disease was hypertension with 53.8% (n = 168), followed by dyslipidaemia, cardiovascular disease and diabetes with 39.4% (n = 123), 27.9% (n = 87) and 23.4% (n = 73), respectively. Regarding clinical findings, values were within the normal range, except for a reduction in mean oxygen saturation of 91.26 ± 5.93 percent and a partial oxygen pressure of 68.68 ± 22.77

Patient Characteristics
The mean age of the patients was <65 years, the majority of them (54.2%) were men. Thirty-six patients (11.5%) were institutionalised prior to hospital admission and the median Charlson comorbidity index score was 1 [0-2]. The most prevalent chronic disease was Healthcare 2023, 11, 1338 5 of 13 hypertension with 53.8% (n = 168), followed by dyslipidaemia, cardiovascular disease and diabetes with 39.4% (n = 123), 27.9% (n = 87) and 23.4% (n = 73), respectively. Regarding clinical findings, values were within the normal range, except for a reduction in mean oxygen saturation of 91.26 ± 5.93 percent and a partial oxygen pressure of 68.68 ± 22.77 mm Hg. In addition, there was an increase in D-dimer with 2892.04 ± 7559.34 ng/mL and C-reactive protein levels of 115.44 ± 165.38 mg/L.

Primary Outcome
In-hospital mortality was 16 The BI has an area under the ROC curve of 0.736. The optimal cut-off point of the BI was 90 points, with a sensitivity and specificity of 70.6% and 71.3%, respectively (Table 2 and Figure 2). Individuals with a BI ≤ 90 points show a 5.95 times higher risk of death than those without baseline functional impairment (Table 2).   To analyse the effect of other variables on the prediction of BI, three multivariate models were generated by a logistic regression; in all of them, the BI was adjusted for other regression covariates. In all models, the confidence intervals for the OR of the BI are narrower than the unadjusted estimate [95% CI: 3.08-11.51] and therefore improve the accuracy of this indicator.
Model 1, consisting of BI, PSI and SatO2, had the highest AUC-ROC (0.804), followed by model 3 (0.792), which consists of BI, age, gender and Charlson Index (Tables 3 and 4). Furthermore, model 1 had the highest sensitivity with 78.4% and model 3 had the highest specificity in the analysis (73.9%). A correct classification of the BI and all the models was observed in terms of the adequate categorisation into the two possible states, surviving or deceased, with values ranging from 83.0 to 84.0% (Table 4).

Secondary Outcome
The predictive ability of the BI is similar to that of the other indexes, and even higher than that of CURB-65 (Table 2). PSI had the best discriminative ability, although A-DROP and BI follow with curves that overlap at some points ( Figure 2). With the exception of CURB-65, all instruments have an AUC-ROC above 0.700. The tool with the highest sensitivity was CURB-65 (88.2%), showing a specificity of 31.0% ( Table 2). The validity of the PSI for its best cut-off point was 72.5% and 76.2% for sensitivity and specificity, respectively. Except for the CURB-65 score, all instruments exceeded the value of 2 for LR+. As for the LR-for the best cut-off points, all of them were below 0.400, which corresponds to the BI (Table 2).

Discussion
All analyses performed in this study indicate that the BI is an adequate predictor of in-hospital mortality due to COVID-19. A BI of 90 was established as the cut-off point for predicting the likelihood of in-hospital mortality.

Sample Characteristics
The sample studied showed characteristics similar to other multicentre studies with large samples. Laosa et al. [24] describe similar data with a median age of 66.06 ± 15.33 years and 55.2% men. Casas-Rojo et al. [20] reported slightly higher data for the median age (69. 40 [56.40-79.90] years) and men (57.2%). They also describe similar data for the prevalence profile of chronic diseases: 50.9% were hypertensive, 39.7% had dyslipidaemia and 19.4% had diabetes mellitus. Therefore, the sample collected in this study may be representative of the profile of patients admitted to Spanish hospitals.
Mortality in our study was 16.4% [95% CI: 12.6-20.9%]. This is lower than that reported in other multicentre studies. Docherty et al. [25] describe a mortality of 26.0% in the UK, Richardson et al. [26] find 21.0% in the New York City Area and Laosa et al. [24] and Casas-Rojo et al. [20] report 19.8% and 21.0%, respectively, in Spain. On the other hand, an Italian study described a mortality of 7.2% [27], which is lower than ours.
In relation to the above, it should be added that some authors have found that being male carries an increased risk of mortality from COVID-19 infection [28,29]. The studies carried out in New York and the UK include more men than in our study (60.3% and 59.9% respectively), and both include patients admitted directly to the ICU, who may have a higher mortality rate due to greater severity. On the other hand, all the studies discussed above were carried out in the first stage of the pandemic and therefore prior to ours, with the clinical management still unknown. On the other hand, the Italian study, although carried out at the beginning of the pandemic and with a higher percentage of men (70%), shows a lower mortality than all of them. When reviewing their results, they stratify mortality by age, with 20.2% in those over 80 years of age. Some authors claim that increasing age increases in-hospital mortality due to COVID-19 [30,31]. On the other hand, a meta-analysis published by Wang et al. [32] describes that the number of deaths attributed to COVID-19 has been disparate in different countries.

The BI as a Predictor of In-Hospital Mortality among COVID-19 Patients
The mean BI in the deceased indicates severe dependence, whereas the mean BI in survivors indicates moderate dependence (p < 0.001). Moreover, the same is true when the BI is analysed in a dichotomised manner using 90 as the cut-off point (≤90 vs. >90). Although the BI was not designed with this objective in mind, it yielded, without adjustment for other covariates, an AUC-ROC of more than 0.700, as well as a very acceptable sensitivity and specificity. We consider these results suitable for two reasons: (a) as stated by Swets [33], a ROC AUC between 0.700 and 0.900 is useful for some specific purposes, and tests with values > 0.900 are considered to have high accuracy; and (b) the BI obtained a similar AUC as the reference scores PSI, CURB-65 and A-DROP. Separately, none of the scores analysed for predicting in-hospital mortality due to COVID-19 achieved an AUC of more than 0.900, and none were clearly better than any of the other ones. The likelihood ratios and accuracy of the BI showed an expected behaviour, with low scores in the deceased and high scores in the survivors. The results confirm that the BI is an adequate predictor of in-hospital mortality. The interest of this result is related to the fact that the BI is simple to calculate, does not require invasive measures and can be completed remotely, e.g., by telephone [34,35]. This makes it possible to screen patients without the need for them to visit a health centre. On the other hand, it is important to note that it is a measure that is routinely obtained in many health centres to monitor patients, mainly for disabilities in the elderly [36,37]. This suggests that healthcare staff are more familiar with obtaining this index than the PSI, CURB-65 and A-DROP.

Adjusting the BI for Prediction of In-Hospital Mortality Due to COVID-19
In our analysis, age was shown to be particularly important. At the end of the followup, the mean age of the deceased was close to 80 years. In contrast, our results did not show statistically significant differences in gender, although the mortality was higher in the men's group (62.7% vs. 37.3%; p = 0.170). In spite of this, we considered it important to adjust the BI for gender, although in most models, significance was lost and the fit did not improve.
Despite the low comorbidity indicated in both groups by the Charlson Index, it is significantly higher in the deceased group. Therefore, adjusting the model for this variable may have helped to dilute the confounding effect of patients with high comorbidities.
On the other hand, although patient institutionalisation showed significance in the bivariate analysis, it led to a confounding behaviour in relation to age, as more than 90% of them were more than 75 years old.
We were unable to find a model that combined all three criteria defined in the methodology: maximum accuracy, avoidance of invasive techniques and inclusion of the variables gender and age together, and therefore three different models were generated. The model that showed the highest accuracy was the one that fitted BI with the PSI scale and SatO2 (model 1), with an OR of 2.65 [95% CI: 1. 24-5.66]. The combination of BI and PSI has been used elsewhere to estimate the mortality risk in CAP [19], with an improved accuracy compared to using each separately. In the present study, it also led to an improvement in the adjustment of the prediction of COVID-19 pneumonia mortality. Model 2 was able to adjust the BI with covariates that did not require invasive techniques: age and SatO2. Finally, although model 3 was the least accurate model, it allowed in-hospital mortality to be established exclusively with baseline variables prior to a COVID-19 infection. The great advantage of the latter model is that it allows establishing a patient's risk before a hypothetical hospital admission. This may be important for the adoption of different strategies, such as prioritisation in vaccination programmes or a selection of individuals for booster doses of the vaccine against COVID-19.

AUC, Sensitivity and Specificity of Models
All models increased the AUC, sensitivity and specificity values compared to the BI alone. Model 1 was the best model: its AUC exceeded 0.8 and it had 78.4% sensitivity and 72.4% specificity. Model 2 had the highest percentage of correctly classified individuals, with an accuracy of 84.0%. Model 3 showed the highest specificity of the three models (73.9%) for correctly classifying survivors. This is important, as it is the only model that can be applied with pre-infection data. Another noteworthy aspect of model 3 is that the probability of mortality in men shows an OR = 2.01, while keeping the rest of the covariates, including the BI, fixed. This is consistent with other studies where mortality in men was found to be higher than in women [7], something we were unable to confirm in the unadjusted analysis.
In the practical application of the different models, it should be noted that: Model 1, while providing the best results, is the most complex to obtain. It requires the use of bloody methods. It also requires the calculation of the PSI. Model 2 is an improvement over model 1, PSI, A-DROP and CURB-65, in that it is not necessary to carry out any bloody method. The only thing that is necessary is that the patient goes to a health centre to obtain the SatO2 or has access to a pulse oximeter at home to measure peripheral saturation. Model 3 is perhaps the one with the greatest advantages over the other models and scales (PSI, A-DROP and CURB-65). This model can be calculated using pre-infection data. In addition, it is not necessary to use bloody methods with the patient and all values can be obtained remotely (e.g., BI via telephone) [34,35].

The BI as a Screening Tool and Its Relationship with Other Prediction Models
The BI has been used as a screening tool for community-acquired respiratory diseases before the COVID-19 pandemic [18,19]. Recently, researchers found an association between BI and COVID-19 mortality. They described that a five-point decrease in the BI increases mortality by 10.0-15.0% [24]. However, they did not specify at what point the BI was obtained-at hospital admission or prior to infection, as was the case in our study. On the other hand, in order to obtain the risk of a patient, one must wait until the patient is infected and hospitalised, and it is not possible to predict mortality before infection, something that our model does allow. Similarly, Wang et al. [38] and da Costa et al. [39] carried out a validation study of the Barthel Index as a predictor of mortality, but in both studies they collected the value of the BI at admission.
Fumagalli et al. [40] developed a prediction model in which they included previous frailty and the BI 2 weeks before infection in institutionalised elderly people ≥ 75 years of age. Although the results they present are acceptable (AUC = 0.870), they use the COVID-19 Mortality Risk Score, a series of variables defined by the authors that are subsequent to the patient's infection and that must be collected on hospital admission.
The combination of risk estimation and an analysis of diagnostic/predictive validity is one of the strengths of this study. We did not find other studies comparing validity (AUC, sensitivity, etc.) specifically for the BI as a tool for predicting COVID-19 mortality.
With respect to the data obtained for the reference indices for mortality prediction (PSI, CURB-65 and A-DROP), our results are in agreement with other studies [8,[41][42][43][44]. The high discriminative power of the PSI may be due to the fact that age and gender are included in the index itself, with a downward correction for younger ages and women. Therefore, models using the BI + PSI should not be further adjusted for these factors.

Limitations
The observational nature of this study is limited to obtaining associations and it cannot establish causality between functional status and mortality. In addition, this is a single-centre study and therefore there may be bias due to population characteristics; although, as noted, the study population is similar to other cohorts used in multicentre studies. We did not analyse the impact of body weight or other variables like the BMI due to the impossibility of obtaining reliable measures of them. Neither was the variable "admission to the ICU" or those patients who were admitted to the ICU from the beginning. At the time of data collection for this study, only patients requiring invasive mechanical ventilation were admitted to the ICU. However, these patients were mainly younger and had a good baseline functional situation. Even though it is highly probable that admission to the ICU could have a bias or impact on mortality, our main result was adjusted by age in our multivariate models 2 and 3. Finally, it should be noted that the use of dichotomous variables simplified the analysis to improve understanding, at the cost of a possible loss of information.

Conclusions
We found a strong association between baseline patient functional status and inhospital mortality due to COVID-19. The BI can be considered a valid score to determine this relationship, showing that only with this instrument and other non-invasive variables prior to admission (age, gender and the Charlson Index), the risk of death in the event of a COVID-19 infection requiring hospital admission can be estimated. These results allow us to discriminate the population that is most likely to die in hospital from COVID-19 before becoming infected, which could help us to make individualised prevention decisions about the general population.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of "Area de Gestión Integrada de Salud de Talavera de la Reina" (protocol code: 30/2020 and date of approval was 4 June 2020).

Informed Consent Statement:
Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.